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The Effect of Performance Feedback on Student Help-Seeking and 
Learning Strategy Use: Do Clickers Make a Difference? 


Abstract 

Two studies were performed to investigate the impact of students’ clicker performance feedback on their help¬ 
seeking behaviour and use of other learning strategies. In study 1, we investigated the relationship between 
students’ clicker performance, self-efficacy, help-seeking behavior, and academic achievement. We found that 
there was a significant positive correlation between their clicker performance and their course grades, and 
help-seeking behavior was negatively and significantly related to clicker and course performance but only for 
participants with high self-efficacy. In study 2, we expanded our focus to determine if participants modified a 
number of learning strategies as a result of receiving clicker performance feedback as well as attempting to 
replicate the clicker-course performance relationship found in study 1. Although participants reported an 
increase in their use of various learning strategies as a result of using the clickers, changes in learning strategy 
use was not significantly related to clicker or term test performance. The relationship between clicker and 
course performance was replicated. The results suggest that clicker-based feedback alone may not be sufficient 
to lead to a successful change in learning strategy use and that students may need more specific instruction on 
self-regulation and effective learning strategy use in order to improve their learning. 

Deux etudes ont evalue l’impact de la retroaction sur la performance des etudiants indiquee par televoteur sur 
leur comportement de recherche d’aide et sur les autres strategies d’apprentissage utilisees. Dans la premiere 
etude, les chercheurs se sont penches sur la relation entre la performance indiquee par televoteur, le sentiment 
d’auto-efficacite, la recherche d’aide et la reussite scolaire. Nous avons trouve une correlation positive 
significative entre la performance indiquee par televoteur et les notes de cours. De plus, nous avons egalement 
decouvert un lien negatif significatif entre le comportement en matiere de recherche d’aide, le televoteur et la 
performance dans le cours, mais uniquement chez les participants ayant un sentiment d’auto-efficacite eleve. 
Dans la deuxieme etude, nous avons elargi notre approche pour determiner si les participants avaient modifie 
plusieurs strategies d’apprentissage apres avoir obtenu une retroaction sur leur performance par televoteur. 
Nous avons de plus tente de repliquer la relation entre le televoteur et la performance dans cours decouverte 
lors de la premiere etude. Bien que les participants aient declare avoir utilise davantage de strategies 
d’apprentissage apres avoir utilise le televoteur, nous n’avons pas trouve de lien significatif entre les 
changements relatifs a ces strategies et le televoteur ou le test de performance de mi-semestre. Nous avons 
replique le lien entre le televoteur et la performance dans le cours. Les resultats suggerent que la retroaction 
offerte par le televoteur n’est pas suffisante en soi pour entrainer un changement fructueux en matiere de 
strategies d’apprentissage et que les etudiants ont besoin d’instructions plus specifiques sur l’autoregulation et 
sur les strategies d’apprentissage efficaces pour mieux apprendre. 
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Clickers have recently been adopted for use in many large university classrooms across 
Canada, the United States, and the United Kingdom (Carnaghan & Webb, 2005; Nicol & Boyle, 
2003). With clickers, students respond via hand-held devices to multiple-choice or true/false 
questions presented by the instructor. The students’ responses are transmitted by radio frequency 
to the classroom computer, where their aggregated responses can be revealed to the class. 
Clickers provide a mechanism for an instructor to receive immediate feedback on learning from 
hundreds of students at once and, more importantly, for students to receive feedback on their 
understanding of the course material (Barnett, 2006). 

The distributors of these devices claim that use of the technology leads to greater student 
engagement and therefore greater classroom learning (Barnett, 2006; Carnaghan & Webb, 2005). 
Both students and faculty report that the experience of using clickers in the classroom is very 
positive and leads to increased classroom satisfaction (e.g., Addison, Wright, & Milner, 2009; 
Davis, 2003; Judson & Sawada, 2002; Kaleta & Joosten, 2007; Morin, Thomas, Barrington, 
Dyer, & Boutchkova, 2009). Clicker use also may increase the use of engaged pedagogies in 
large classes by enabling widespread anonymous participation (Caldwell, 2007; Morin et al., 
2009; Rhem, 2009), providing immediate feedback, and/or creating a cumulative record of 
participation (Beatty, 2004). One of the critical elements and most commonly cited benefits of 
clickers is the immediate feedback students receive about their understanding of the course 
material (Barnett, 2006). However, little is known about how students respond to that feedback. 
Do they alter their learning strategies to improve their understanding of the course material? If 
so, how? 

We performed two studies to examine these questions more fully. The first concentrated 
on one learning strategy, students’ help-seeking behaviour, whereas the second focused on a 
range of learning strategies. 


Study 1 

This study had several objectives. First, we wanted to determine the relationship between 
students’ clicker performance and their course grades, in order to detennine the diagnostic value 
of clicker performance in academic performance. We hypothesized that there would be a positive 
relationship between students’ clicker responses and their grades in the course and that the 
relationship would be evident when participants’ past academic performance, clicker 
participation (i.e., how often they responded to clicker questions), learning efficacy, and sex 
were controlled for. This was done to demonstrate that clicker performance is related to course 
performance and that the relationship is not simply a by-product of the other variables. We also 
wanted to demonstrate that students’ clicker responses represent an effort to answer questions 
correctly, rather than arbitrarily pressing buttons to receive a participation grade, as has been 
suggested by some who question the pedagogical utility of clickers. 

Second, we wanted to explore the relationship between students’ clicker performance and 
their help-seeking behaviour. One assumption that underlies the use of clickers is that upon 
receiving perfonnance feedback, students will seek help if they see that their comprehension of a 
topic has failed (Bruff, 2009). Yet, to our knowledge, no research has investigated whether or not 
this is true. 

We anticipated that the clicker performance-help-seeking relationship would be 
moderated by the students’ sense of self-efficacy—their beliefs about their own ability to achieve 
goals (Svinicki, 2004). Although students may have a general sense of self-efficacy about their 
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ability to learn in university, they may also have a sense of self-efficacy that is task specific 
(Svinicki, 2004), which is why we examined students’ sense of self-efficacy in introductory 
biology specifically. Researchers have suggested that students who do not believe they are 
capable of completing a task, those with low self-efficacy, are less likely to engage in self- 
regulated activities such as seeking help and less likely to persist at tasks (Pintrich & de Groot, 
1990). For students with low levels of self-efficacy, we did not expect that their clicker use 
would lead them to seek help even if needed. 

Locus of control was another individual difference variable we examined. Students who 
have an internal locus of control believe that success on tasks is influenced by effort, motivation, 
and ability (Findley & Cooper, 1983). In the face of failure, they will critically examine what 
they need to do to improve their performance the next time, including potentially seeking help. 
Those with an external locus of control attribute performance to luck, timing, or other variables 
over which they have no control and thus are not motivated to make changes in their learning 
strategies, such as help seeking, in the face of failure. 

Consistent with research by Karabenick and Knapp (1991), we also predicted that when 
students do seek help, it would be from informal rather than formal sources (e.g., friends rather 
than the instructor), even though it is likely that the fonnal sources would be better equipped to 
provide effective assistance. 

Finally, we investigated participants’ attitudes towards the use of clickers in class and 
their relation to clicker and academic performance. A number of researchers (e.g., Davis, 2003; 
Judson & Sawada, 2002; Kaleta & Joosten, 2007) have found that students react positively to 
clicker use and we wanted to detennine if their attitudes were related to how well they perform 
in answering clicker questions or in the course. Based on our experience with clickers, we 
predicted that students, regardless of their clicker or course performance, would be positive 
about their use. 

Method 

Participants. Three-hundred and twenty-four students (234 women) enrolled in a full- 
year introductory biology course at a large medical doctoral university in Canada participated in 
the study, representing a 27% response rate. The majority of participants were in the first year of 
their academic programs (96%) and in the Faculty of Science (68%), with a sizable minority in 
the Faculty of Health Sciences (26%). 

Measures. Participants completed a 21-item survey. Of the 21 items, 18 were developed 
by the researchers for this study. The three remaining items were multifaceted and taken from 
standardized measures. All of the items are addressed below. 

Demographic items. Four items assessed participants’ sex, program year, faculty, and use 
of clickers in other courses. 

Help-seeking behaviour. Participants’ help-seeking behaviour was assessed using two 
measures. They completed the Help-Seeking scale of the Motivated Strategies for Learning 
Questionnaire (MSLQ) (Pintrich, Smith, Garcia, & McKeachie, 1993), indicating, on a 5-point 
scale (0 = not at all true of me, 4 = very true of me), the extent to which they seek help from their 
peers or instructors. Cronbach’s alpha, which measures the degree of internal consistency 
amongst items on a scale (Cronbach, 1951), was .63 for the 3-item version used in the current 
study. 
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Participants also indicated, on a 5-point scale (0 = never to 4 = all the time), how often 
they had accessed 11 different sources of help since the beginning of the academic year. 
Specifically, they were asked to rate how often they turned to a teaching assistant (TA), the 
course instructor, their parents, their friends, the Internet, study groups for the course, the library, 
campus-based learning skills services, residence staff, and sophs (upper-year students) for 
residence- and nonresidence-based students. The participants’ ratings on help-seeking resources 
were subject to a principal components analysis to determine their factor structure. One factor 
(Cronbach’s alpha = .75), referred to here as the Helping Resources factor, was extracted. The 
MSLQ Help-Seeking subscale and the Helping Resources factor were significantly correlated 
(r = .40,/? < .001). 

Self-efficacy . Participants completed the eight-item Self-Efficacy scale of the MSLQ 
(Pintrich et al., 1993), which is designed to detennine participants’ judgment of their ability to 
perform a task. Participants indicated, on a 5-point scale (0 = not at all true of me, 4 = very true 
of me), their self-efficacy in their introductory biology course. The MSLQ has been found to be a 
reliable and valid measure of learning strategies (see Pintrich et al., 1993, for a review). For the 
current investigation, Cronbach’s alpha for the scale was .95. 

Locus of control. Participants’ academic locus of control was assessed with a 13-item 
version of the Academic Locus of Control scale (Trice, 1985). Trice (1985) provides information 
on the reliability and validity of the original 28-item true/false version of the scale. In the current 
study, participants rated each item on a 5-point scale (0 = not at all true to 4 = very true). Six of 
the 13 items were worded to reflect an external locus of control and five were worded to reflect 
an internal locus. For all analyses, the five internal items were recoded to reflect an external 
locus of control. Cronbach’s alpha for the 13-item version was .68. 

Academic achievement. Participants were asked to give researchers permission to access 
their admission average (i.e., their high school average used as the basis for their admission to 
the university), test and final course grades in introductory biology, and clicker responses. 
Pennission was received from 80% to access their admission average, 69% for course grades, 1 
and 75% for clicker responses, respectively. Clicker performance was calculated by dividing an 
individual participant’s number of correct responses by the total number of questions answered. 
The overall mean clicker perfonnance for this study was .47 (SD =.11). 

Perceptions of clickers. Participants completed six items assessing their perceptions of 
the effect of the clickers on various characteristics. They rated the extent to which they perceived 
that the clickers affected their learning of the course material, enthusiasm for the course, course 
attendance, attention in class, and interest in taking further courses in biology. Participants also 
rated their feelings about the clickers. All of the ratings were on a 5-point scale, with higher 
ratings indicating greater perceived learning, greater enthusiasm, etc. 

Procedure. Students enrolled in two sections of a full-year introductory biology course 
used clickers in their classes. Both sections were taught by the same three instructors, in series. 
All three instructors used clickers in their classes. The number of questions asked ranged from 
one to six, depending on the class. The types of questions asked varied from factual to 
conceptual and were used to assess both prior knowledge and current understanding of concepts. 
Participation, in the form of responding to at least 80% of clicker questions, was worth 5% of the 
course grade. 


Participants in Study 1 scored approximately 4% higher on three term tests and the final examination than the 
classes overall. This difference will be addressed in the General Discussion of the paper. 
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Seventeen weeks into the course, students received an e-mail invitation to participate in 
an online survey. A reminder was sent one week later. Interested participants clicked on a 
hyperlink in the e-mail to access the survey. To be entered into a draw for one of three Apple 
iPod Shuffles (or gift certificates of equivalent value), participants were required to input their 
student number at the end of the survey. Upon completing the survey, participants clicked on a 
button to submit their responses. Submission of the survey was taken to indicate consent to 
participate. 

Results 

The first objective of the study was to establish the relationship between students’ clicker 
performance and their course grades, and a significant correlation was in fact found between the 
two variables (r= .63, p < .001). A hierarchical multiple regression was performed to examine 
the clicker performance-grades relationship, partialling out students’ admission average, 
learning self-efficacy, science self-efficacy, sex, and clicker participation (i.e., number of clicker 
questions answered). Even partialling out these variables, the clicker perfonnance-grade 
relationship was significant (see Table 1). 


Table 1 

Correlations Between Predictors and Final Grade 


Predictors 

Zero-order 

conelation 

Partial correlation 

/test 

Admission average 

.58* 

.34 

4.78* 

Self-efficacy 

.62* 

.37 

5.21* 

Science efficacy 

.41* 

-.01 

-.11 

Sex 

-.19 

-.04 

-.58 

Clicker participation 

.35* 

.29 

4.01* 

Clicker performance 

.65* 

.47 

7.00* 


Note. The partial correlations presented are those after all of the predictors have been entered 
into the equation in the order presented in the table (i.e., clicker performance last). 

*p < .001. 

The second objective of the research was to explore the relationship between participants’ 
help seeking, clicker performance, and academic performance. Clicker perfonnance was not 
significantly related to either the Helping Resource variable (r = -.08, ns) or the MSLQ Help- 
Seeking subscale (r = -.01, ns), nor were course grades (r = -.12, ns and r = -.01, ns, 
respectively). 

To determine if self-efficacy moderated the help-seeking-clicker perfonnance and help¬ 
seeking-grade relationships, we performed a tertile split on participants’ self-efficacy scores and 
examined the correlations separately. Based on the tertile split, participants with low self- 
efficacy had a score of 2.12 or less, and those with high self-efficacy had a score of 3.01 or 
greater (on a scale from 0 to 4). 

For participants with low self-efficacy, the correlations between help seeking and clicker 
performance were not significant (r = .11, ns, for Helping Resources and r= .12, ns, for the 
MSLQ Help-Seeking subscale). On the other hand, for participants with high self-efficacy, there 
was a small but significant correlation between clicker perfonnance and Helping Resources (r = - 
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.27, p < .05), but not the MSLQ Help-Seeking subscale (r = -.17, ns). We performed a Fisher 
r-to-z transfonnation to determine if the Helping Resources-clicker perfonnance correlations 
were significantly different between participants with low and high self-efficacy. The correlation 
for participants with high self-efficacy was significantly larger than that of their low self-efficacy 
counterparts (z = 2.28 ,p< .05). 

A similar pattern of results was evident with the help-seeking-course grade relationship. 
For participants with low self-efficacy, the correlations were not significant (r =.19, ns, for 
Helping Resources and r= .03, ns, for the MSLQ Help-Seeking subscale). On the other hand, 
participants with high self-efficacy had significant and negative correlations between grades and 
Helping Resources (r = -.38, < .001) and the MSLQ Help-Seeking subscale (r = -.41, 

p < .001). We performed Fisher r-to-z transfonnations to determine if the help-seeking-clicker 
performance correlations were significantly different between participants with low and high 
self-efficacy. The correlation for participants with high self-efficacy was significantly larger than 
that of their low self-efficacy counterparts for both Helping Resources (z = -2.34, < .01) and 

the MSLQ Help-Seeking subscale (z = -3.43,/? < .001). 

We also examined self-efficacy differences in where participants sought help. 
Participants with high self-efficacy were more likely to seek help from their instructor than were 
their low self-efficacy counterparts [/(181) = -3.27, p < .005; M= 1.17, SD = 1.10 and M = .75, 
SD = .71, respectively]. Participants with low self-efficacy were more likely to seek help from 
the library than were their high self-efficacy counterparts [t(181) = 3.61,/><.001; M= 1.81, 
SD= 1.42 and M= 1.10, 577=1.38, respectively]. There were no significant self-efficacy 
differences for the other nine individual helping resources. 

To determine if locus of control moderated the help-seeking-clicker perfonnance and 
help-seeking-course grades relationships, we performed a tertile split on participants’ locus of 
control scores and examined the correlations separately. Based on the tertile split, participants 
with an internal locus of control had a score of 1.31 or less, and those with an external locus of 
control had a score of 1.85 or greater (on a scale from 0 to 4). For participants with an internal 
locus of control, the correlations between help seeking and clicker perfonnance were not 
significant (r = -.15 and -.16, ns, for Helping Resources and the MSLQ Help-Seeking subscale, 
respectively) nor were they significant for participants with an external locus of control (r = .03 
and .09, ns, for Helping Resources and the MSLQ Help-Seeking subscale, respectively). 
Similarly, for participants with an internal locus of control, the conelations between help seeking 
and course grades were not significant (r = -.21, ns, for Helping Resources and r = -.04, ns, for 
the MSLQ Help-Seeking subscale), nor were they significant for participants with an external 
locus of control (r = .24 and .20, ns, respectively). 

As can be seen in Table 2, when participants sought help, they sought it from their friends 
significantly more often than any other resources [e.g., more than the Internet, /(322) = 11.38, 
p < .001, or the biology TA, t( 322) = 20.83, p < .001]; in other words, they were more likely to 
seek help from infonnal rather than formal sources. It is noteworthy that participants were almost 
equally likely to seek help from their parents and the course instructor [/(323) = 1.72, ns]. 
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Table 2 


Resources Studen ts Accessed for Help 


Resource 

M 

SD 

Friends 

3.13 

.98 

Internet 

2.32 

1.18 

Biology TA 

1.62 

1.15 

Biology study groups 

1.57 

1.46 

Fibrary 

1.44 

1.45 

Parents 

1.15 

1.29 

Biology instructor 

.99 

.99 

Sophs (upper-year students) 

.89 

1.18 

Residence staff 

.77 

1.14 

Teaming skills services 

.57 

.95 

Office-campus dons (peer 
mentors) 

.23 

.66 


The third objective was to examine participants’ perceptions of the clickers and their 
perceptions’ relationship with clicker and course perfonnance. Overall, participants were very 
positive about using clickers in the classroom. Specifically, 89% of participants indicated that 
they were moderately or extremely positive about the use of clickers and 86% felt that the 
clickers somewhat or greatly facilitated their learning of the course material. A majority of 
students also felt that the clickers somewhat or greatly increased their enthusiasm for the course 
(65%), attention in class (62%), and attendance (60%). Even though students’ perceptions of 
clickers were positive, a majority indicated that clicker use had no effect on their interest in 
pursuing further education in the biological sciences (78%). Participants’ attitudes towards the 
clickers were not related to their clicker performance (r ranged from -.12 to .15, ns) or their final 
course grade (r from -.14 to .16, ns). 

Discussion 

As predicted, there was a significant correlation between participants’ grades and their 
clicker performance, which was maintained when we partialled out past academic achievement, 
self-efficacy, sex, and clicker participation. Thus, clicker performance has diagnostic value for 
students’ performance on their exams, over and above the variables outlined above (i.e., the 
relationship is not simply a product of those variables). This is a particularly important finding, 
as instructors can advise their students with confidence of the clicker performance-course grade 
relationship and recommend, if they are not doing well on the clicker questions, that they may 
need to make adjustments to their studying to ensure that they perform well on tests. Also the 
correlation suggests that students were not just arbitrarily pressing buttons to receive the 
participation grade, but were making a serious effort to answer the questions correctly. For 
instructors, this is important, as some may incorrectly believe their students are responding 
arbitrarily. 

The diagnostic value of students’ clicker performance is, of course, influenced by the 
degree of congruence of clicker questions with test questions. Clicker questions and test 
questions used in the classes under investigation were generally very similar in their multiple- 
choice structure and content focus. 
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Contrary to prediction, there was no overall significant relationship between help seeking 
and clicker performance. However, we did find that a small but significant negative relationship 
does exist for participants with high, but not low, self-efficacy. Students who felt that they could 
do well in the course tended to seek help if they were not doing well on the clicker questions. 
Like all the participants, they tended to turn to their friends for help. Despite the fact that 
working with friends is often encouraged at university, this is only likely to be helpful if peers 
are able to assist their friends successfully. Furthermore, students were equally inclined to ask 
their parents as their instructor for help (although participants with high self-efficacy were more 
likely to seek out the instructor for help than were their counterparts with low self-efficacy). As a 
strategy for success, asking parents for help with biology may not be the most prudent choice. 
These results support the research of Karabenick and Knapp (1991), who also found that students 
were likely to seek help from informal sources, such as friends. The second source of help 
participants sought out was the Internet. Being able to distinguish reliable from unreliable 
sources on the Internet is dependent on information literacy skills and may be a challenge to 
many students. 

Locus of control was not found to be significantly correlated with help seeking and 
clicker perfonnance, nor was it predictive of grades. Research by Findley and Cooper (1983) 
suggests that college students may have a restricted range of locus of control scores compared to 
all individuals in their age group, which makes it less likely to be predictive of academic 
achievement. Similarly, in the present study, the range of scores for locus of control was also 
restricted, and this may be why this variable was not a significant predictor of the outcome 
variables. 

Also, there was no overall significant relationship between help seeking and course 
grades. However, we did find that a significant negative relationship exists for participants with 
high, but not low, self-efficacy. It seems possible that students who felt that they could do well in 
the course, but had not done well in the first half of the course, were the ones who tended to seek 
help. The help-seeking data was collected 17 weeks into the course and participants would have 
already accumulated a substantial portion of their final grade. They may have sought help based 
on poor performance on earlier tests and the help (e.g., from friends) may not have resulted in 
substantial improvement in their performance. 

Consistent with previous research (e.g., Caldwell, 2007; Kaleta & Joosten, 2007), 
participants reported that clickers increased their enthusiasm for class, facilitated their attention 
in class and their learning of their course material, and led to an increase in attendance. Overall, 
participants reported being positive about their clicker use in biology, and that positive 
perception was not dependent upon clicker or course performance (i.e., they did not feel positive 
because they were doing well on the clicker or test questions). We believe that this is an 
important finding, as science educators often struggle to find ways to motivate their students 
(Brewer, 2004; Dillon & James, 1977) and clickers may help serve this purpose. 

In Study 1, we confirmed that there is a significant relationship between students’ clicker 
record and course performance. We also confirmed that students with high self-efficacy tended 
to seek help if their clicker performance was in need of improvement and those that sought help 
tended not to have performed as well in the course overall. Finally, we found that students 
perceived the clickers positively, independent of their clicker or course perfonnance. However, 
there were still a number of important questions that remained unanswered. One of the foci of 
the first study was the relationship between help seeking and clicker and course performance, but 
help seeking is only one of an array of learning strategies in which students may engage. If most 
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students (except for those with high self-efficacy) do not tend to seek help as a result of their 
clicker perfonnance, is it possible that their clicker performance is influencing their use of other 
learning strategies and, if so, how? 

Also, in Study 1, there were three instructors, who may have used the clickers in different 
ways. This was recognized as a possible confounding factor. Therefore, it was important that in 
the second study, the data be collected earlier in the semester, with students being taught by only 
one instructor. This would help ensure that the form and manner of the clicker questions would 
be consistent prior to the survey. 


Study 2 

Introduction 

Even though participants in Study 1 did not generally seek help based on their clicker 
performance (except for those participants with high self-efficacy), it does not mean that they 
were not processing the perfonnance feedback and adjusting their learning strategies to improve 
their performance in the course. Barnett (2006) found that students indicated that receiving 
feedback on their comprehension was one of the most common benefits of using clickers. To 
examine if and how students are using this feedback to facilitate their learning, we examined 
whether students changed their learning strategies based on their clicker feedback and what form 
those changes took. In particular, we expanded the learning strategies under investigation to 
include rehearsal, elaboration, organization, self-regulation, effort regulation, and critical 
thinking, not just help seeking. 

We also examined the moderating role of students’ use of metacognitive self-regulation 
in the relationship between clicker performance and use of learning strategies. The literature 
supports the importance of metacognitive self-regulation to academic success (Svinicki, 2004). 
Students’ ability to monitor their own cognitions enables them to successfully change strategies 
when it is apparent that their understanding has failed (Pintrich & de Groot, 1990). This may be a 
particular issue for first-year students, whose metacognitive skills are generally not as 
sophisticated as those of more senior students (Pintrich & Zusho, 2007). 

Another difference from Study 1 is the timing of the research. Study 2 was conducted 
6 weeks, rather than 17 weeks, into the course. During this period there was only one instructor 
and he asked questions in a consistent manner. The instructor primarily used the “peer 
instruction” method in asking clicker questions. Peer instruction is a variation on the “think-pair- 
share” active learning method (Rhem, 2009), in which students first think about the question and 
answer individually using the clickers. Next they partner with a classmate to discuss the question 
and respond to the same clicker question with their newly derived consensus answer. This 
method was chosen because researchers have found that students who engage in interactive 
discussion in the classroom, when each individual is accountable initially for their own response, 
leads to greater attention being paid in class and students making more sense of difficult 
concepts (e.g., Nicol & Boyle, 2003). In fact, Knight and Wood (2005) found that there were 
significant gains in learning and conceptual understanding when biology students have the 
opportunity to engage in peer instruction in class. Preszler, Dawe, Shuster, and Shuster (2007) 
suggested this may be particularly important in science programs, as students have been found to 
graduate without having a truly meaningful understanding of their discipline. They postulated 
that it is only when the use of clickers is paired with student-centred forms of learning, such as 
peer instruction, that clickers are likely to enhance learning. Furthermore, they suggested this 
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may be why not all studies show gains in actual learning as a result of the use of clickers in class. 
The lack of consistency of instruction may be one reason that clickers had little impact on help 
seeking for the majority of participants in Study 1. Also, the differential use of the clicker 
questions may have had an impact on the clicker performance-course performance relationship 
as well as the relationship between students’ perceptions of the clickers and their clicker and 
course performance. 

Another difference between clicker use in studies 1 and 2 is that all students were 
informed of their cumulative clicker perfonnance by means of two e-mails sent prior to the first 
tenn test. The e-mails also provided information on how to interpret their cumulative clicker 
performance and a list of on-campus learning resources they could turn to if they needed help. 
We examined whether receiving this e-mail specifically led students to change their learning 
strategies. 

Method 

Participants. Two hundred and sixty-one students (174 women) enrolled in a full-year 
introductory biology course at a large medical doctoral university in Canada participated in the 
study, a 19% response rate. The majority of participants were in the first year of their academic 
programs (98%) and in the Faculty of Science (70%), with a sizable minority in the Faculty of 
Health Sciences (18%). 

Measures. Participants completed a 20-item survey. Eighteen of the items were 
developed for this study; the other two were multifaceted and taken from standardized measures. 
All of the items are addressed below. 

Demographic items. Four items assessed participants’ sex, program year, faculty, and use 
of clickers in other courses. 

Clickers’ effect on learning strategy use. Participants indicated how much using clickers 
in their biology course changed their use of learning strategies by completing an adapted version 
of the Learning Strategies scales of the MSLQ [Pintrich et al., 1993; responding on a scale of 1 
(greatly decreased) to 5 (, greatly increased ); see Table 3 for information on the Learning 
Strategies scales]. 


Published by Scholarship@Western, 2010 



The Canadian Journal for the Scholarship of Teaching and Learning, Vol. 1, Iss. 1 [2010], Art. 6 


Table 3 


Number of Items, Cronbach’s Alphas, and Sample Items on the Revised MSLQ Learning- 
Strategies Scales _ 


MSLQ scale 

Items 

Alpha 

Sample item 

Rehearsal 

4(4) 

.73 (.69) 

“Say the material to yourself over and over 
when studying for this course” 

Elaboration 

5(6) 

.79 (.75) 

“Relate the reading material to what you 
already know” 

Organization 

4(4) 

.75 (.64) 

“Outline the material to help organize your 
thoughts when studying for the course” 

Critical thinking 

4(5) 

.72 (.80) 

“Think about possible alternatives whenever 
you read or hear an assertion or conclusion in 
this course” 

Self-regulation 

5(12) 

.76 (.79) 

“Make up questions to help focus your reading 
for the course” 

Time management 

3(8) 

.56 (.76) 

“Make good use of your study time for this 
course” 

Effort regulation 

2(4) 

.63 (.69) 

“Manage to keep working until you finish even 
when course materials are dull or 
uninteresting” 

Peer learning 

3(3) 

.61 (.76) 

“Work with other students from this course to 
complete course assignments” 

Help seeking 

3(4) 

.62 (.52) 

“Ask the instructor to clarify concepts you do 
not understand well” 


Note : The number of items and alphas for the original scales of the MSLQ are provided in 
parentheses. 


Clicker performance. Participants also completed three other items to indicate the effect 
they perceived that their clicker performance had on their study habits. Two items asked if their 
clicker performance influenced how they prepared for their biology lectures and tests and, if so, 
how. The third item asked if the e-mails that they received in weeks 3 and 5 of the course 
motivated them to change their study habits or seek academic help and, if so, what changes they 
made and why. 

Metacognitive self-regulation. Metacognitive self-regulation was assessed using five 
items from the 12-item Metacognitive Self-Regulation scale of the MSLQ (Pintrich et ah, 1993). 
Participants responded on a 5-point scale from 0 {not at all true) to 4 {very true). Cronbach’s 
alpha for the five-item version was .71. 

Help-seeking behaviour. Participants indicated, on a 5-point scale from 0 (n ever) to 4 
{all the time), how often they had accessed 10 potential sources of help with their biology course. 
Specifically, they were asked to rate how often they turned to a biology TA, the course 
instructor, their parents, their friends, the biology course Web page, other Internet resources, 
biology study groups, the library, on-campus learning skills services, and other resources. The 
participants’ ratings on help-seeking resources were subject to a principal components analysis to 
determine their factor structure. One factor, referred to here as the Helping Sources factor 
(Cronbach’s alpha = .77), was extracted. This factor, comprised nine of the 10 sources of help 
(the “Other” category was dropped). 
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Perceptions of clickers. Participants rated the extent to which they perceived that the 
clickers affected their learning of the course material and their feelings about the clickers. All of 
the ratings were on a 5-point scale, with higher ratings indicating greater perceived learning, 
more positive feelings, etc. 

Other clicker questions. Participants were also asked how often they try to answer the 
clicker questions correctly (0 = never to 4 = always). 

Academic achievement. Participants were asked to give the researchers permission to 
access their admission average (i.e., their high school average used as the basis for their 
admission to the university), their test 2 and final course grades in introductory biology, and 
clicker responses. Pennission was received from 81% for access to their admission average, 69% 
for course grades, and 73% for clicker responses. As in Study 1, clicker perfonnance was 
calculated by dividing an individual participant’s number of correct responses by the total 
number of questions answered. The overall mean clicker performance before the first tenn test 
was .54 (SD = .14). 

Procedure. Students involved in the study were enrolled in two sections of a year-long 
biology course. Six weeks into the course, they received an e-mail invitation to participate in an 
online survey. A reminder was sent one week later. Interested participants clicked on a hyperlink 
in the e-mail to access the survey. To be entered into a draw for one of three Apple iPod Nanos 
(or gift certificates of equivalent value), participants were required to input their student numbers 
at the end of the survey. Upon completing the survey, participants clicked on a button to submit 
their responses. Submission of the survey was taken to indicate consent to participate. In each 
class the instructor included two to six clicker questions and peer instruction was the primary 
method of clicker-based instruction used. 

Results 


Similar to the first study, clicker perfonnance before the first term test was a significant 
predictor of test performance. Specifically, the proportion of the participants’ correct clicker 
responses was positively and significantly correlated with their tenn test score (r = .51, 
p < .001). A hierarchal multiple regression analysis was performed to examine the clicker 
performance-grades relationship, partialling out participants’ admission average, sex, and clicker 
participation (i.e., number of clicker questions answered). Participants’ clicker performance 
preceding the first-term test continued to be a significant predictor of test perfonnance (see 
Table 4). 


2 

“ Participants in Study 2 scored approximately 5% higher on first term test than the classes overall. This difference 
will be addressed later in the General Discussion of the paper. 
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Table 4 

Correlations Between Predictors and Term Test Grade 


Predictors 

Zero-order 

correlation 

Partial correlation 

t test 

Admission average 

.53* 

.39 

5.08* 

Sex 

-.10 

-.06 

-.66 

Clicker participation 

.29* 

.12 

1.46 

Clicker performance 

.51* 

.33 

4.21* 


Note. The partial correlations presented are those after all of the predictors have been entered 
into the equation in the order presented in the table (i.e., clicker performance last). 

* p < .001. 

We examined whether participants reported changing their learning strategies as a result 
of using clickers and found that 85% reported an increase in their use of at least one of the eight 
learning strategies. As can be seen in Figure 1, over 50% of students indicated that they 
increased their use of rehearsal, elaboration, metacognitive self-regulation, and effort regulation 
based on clicker use in the course. Over 40% reported increased use of critical thinking and 
organization, whereas a minority reported an increase in their use of peer learning and help 
seeking based on clicker use in the course. A repeated-measures ANOVA was performed to 
examine differences between participants’ reported increase in the use of the eight learning 
strategies [F(7, 227)= 19.7, /> = .001]. Post hoc analyses revealed that participants reported 
increasing their use of metacognitive self-regulation significantly more, and their use of peer 
learning significantly less, than they reported increasing their use of other strategies. They also 
reported increasing their use of help seeking significantly less than they reported increasing 
rehearsal, elaboration, and effort (all ps < .05). 


3 A Bonferroni correction was employed to adjust for multiple comparisons. 
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% Respondents 



■ Greatly Increased 
Somewhat Increased 


Figure 1. Learning strategy use increase as a result of clickers. 

Participants also indicated if their performance on the clicker questions influenced how 
they prepared for the biology lectures and exams. Only 23% of participants indicated that their 
performance on clicker questions influenced how they prepared for biology lectures, whereas 
62% indicated that their performance influenced how they prepared for the first term test. 

We also examined whether receiving directed feedback by means of the e-mail on their 
clicker record might lead students to change their learning strategies. Although only 20% of 
students reported changing their learning strategies based on the e-mail, the group that changed 
their strategies had significantly poorer clicker performances (M= .42, SD = .202) than the group 
that did not make a change [M=.51; SD = .225; t(183) = -2.10 ,p = .05]. 

None of the other indicators of change in learning strategies was significantly related to 
clicker performance. Participants’ clicker performance was not significantly correlated with their 
changes in learning-strategy use as assessed by the eight revised MSLQ scales (Pintrich et al., 
1993; see Table 5). Participants who reported changing either their lecture or test preparation did 
not differ significantly on their clicker performance from those who did not report making these 
changes [7(184) = -1.02, ns, for lectures and t(184) = .90, ns, for term tests]. 

Finally, we investigated participants’ use of metacognitive self-regulation. Participants 
who reported changing their lecture preparation had significantly higher self-regulated learning 
scores than those who indicated no change [7(246) = 3.12, p< .005; M=2.64 (SD = .75) and 
M= 2.30 (SD = .74), respectively]. Furthermore, participants’ metacognitive self-regulation was 
significantly related to their increase in the use of five of the eight learning strategies assessed by 
the revised MSLQ (see Table 5). However, participants who reported changing their preparation 
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for the term test did not differ significantly on their metacognitive self-regulated learning scores 
from those who indicated making no change [7(245) = 1.43, ns). 

Neither the participants who reported changing their lecture preparation, nor those who 
reported changing their preparation for the tenn test, differed significantly on their tenn test 
grade [/(170) = 1.32, ns, and /(170) = -.24, ns, respectively] from their nonchanging counterparts. 
Finally, participants’ changes in learning strategy with the revised MSLQ were not significantly 
related to their performance on the first term test, with one exception: rehearsal (see Table 5). 

Table 5 


Correlations Between Revised MSLQ Learning Strategies and Clicker Performance, Self- 
Regulated Learning, and Term Test Grade _ 


Revised MSLQ 

Clicker 

Self-regulated 

Term test grade 

learning strategies 

performance 

learning 


Rehearsal 

-.16 

.15 

-.25* 

Elaboration 

-.04 

.34* 

-.03 

Organization 

-.10 

.29* 

-.13 

Critical thinking 

.01 

.30* 

-.05 

Self-regulation 

-.09 

.25* 

-.07 

Effort regulation 

-.07 

.24* 

-.06 

Peer learning 

.02 

.20 

.02 

Help seeking 

-.11 

.18 

-.11 

*p < .001. 




Consistent with Study 1, participants’ help seeking as 

assessed by the Helping Sources 

factor was not significantly related to clicker performance (r = 

= -.2, ns) or term test performance 

(r = -.17, ns). Also, participants sought help with their biology course from friends more than 
any other resource, such as the biology course Web page [/(256) = 2.04, p < .05] or the biology 

TA [7(255) = 12.03, p 

< .001] (see Table 6). Contrary to Study 1, participants were significantly 

more likely to seek out their instructor than their parents for help with the course [/(256) = 6.85, 
p < .001] (see Table 6), but seeking out the instructor was still rated as rare (1.19 on a scale of 0 


to 4). 

Table 6 

Sources Students Accessed for Help 


M 

SD 

Friends 

2.64 

1.18 

Biology Web page 

2.44 

1.16 

Other Internet 

1.71 

1.31 

TA 

1.47 

1.21 

Biology study group 

1.30 

1.42 

Library 

1.19 

1.28 

Instructor 

1.02 

1.12 

Learning skills 

.53 

.98 

Parent 

.48 

.96 
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The vast majority of students (95%) reported that they “always” or “almost always” tried 
to answer the questions correctly, despite the fact that only 54% of the clicker questions were 
answered correctly over the 6-week period. 

Two items were repeated from Study 1 to detennine participants’ perceptions of the 
clickers. As in Study 1, 87% of participants felt that the clickers somewhat or greatly facilitated 
their learning of the course material and 84% felt positive about using clickers in their course. 
These two items were not significantly related to either participants’ clicker or tenn test 
performance (r ranged from .01 to .19, ns). 

Discussion 

As in Study 1, clicker performance was a significant predictor of achievement, in this 
case of term test grades, even when controlling for prior academic performance, clicker 
participation, and sex effects. This finding, in conjunction with participants’ reports that they 
attempt to answer clicker questions correctly, demonstrates that students do take responding to 
clicker questions seriously and do not simply respond arbitrarily to earn the participation grade. 

Most students reported changing their learning strategies based on their use of clickers 
and their reported increases in strategy use was related to their metacognitive self-regulation for 
five of the eight strategies assessed. Interestingly, those reported changes were not related to 
their actual clicker performance or their performance on the term test (except for changes in 
rehearsal use). This may suggest that even though participants reported increases in the use of 
several different learning strategies, they were not necessarily skilled in selecting or properly 
employing the strategies that would lead to increased test scores. In fact, we found that the 
increased use of rehearsal was negatively correlated with term test performance. That is not 
surprising, as Svinicki (2004) and others (e.g., Biggs & Tang, 2007) have noted that learners who 
use repetition as a strategy also tend to take a more shallow approach to learning, which leads to 
poorer recall. This means that, even though we might want to have learners change or increase 
their strategy use as a result of their clicker feedback, we also want them to be more focused on 
which strategies they select, as not all strategies are well suited to deep learning. As Biggs and 
Tang (2007) indicate, convincing students not to use rehearsal as the dominant learning strategy 
can be difficult, as it is easier than others, such as elaboration or organization). The apparent 
inability of many students to translate clicker feedback into improved academic performance is 
reflected in the research of Addison, Wright, and Milner (2009), who found that only high- 
achieving students showed academic improvement after the introduction of clickers. 

Those participants who reported increasing their use of one or more learning strategies 
tended to increase metacognitive self-regulation, effort regulation, rehearsal, elaboration, 
organization, and/or critical thinking, but few had increased their use of peer learning or help 
seeking. Their lack of use of help seeking, even when e-mailed a list of learning resources , is 
consistent with Study 1 and suggests that simply infonning students of what resources are 
available does not necessarily motivate them to avail themselves of those resources. Also 
congruent with Study 1 is that participants reported seeking help from their friends more often 
than formal sources of help, such as the course instructor, TA, or learning skills services. Despite 
the fact that friends were the number one-cited source for help, the biology course Web page 
was second, which is promising, as it is an excellent resource. It may be that students seek help 
from resources that are perceived to be less intimidating, less ego threatening, or more 
convenient. 
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It was noteworthy that the 20% of participants who reported changing their learning 
strategies as a result of receiving the e-mail had significantly poorer clicker records than those 
who did not. This suggests that there may be some benefit to adopting the e-mail intervention, at 
least for the poorer performing students. Providing more explicit infonnation about the 
diagnostic value of clicker performance and how to use effective learning strategies to improve 
learning of the material may be necessary to make the clicker feedback and e-mail intervention 
lead to improved learning. 

Finally, also consistent with Study 1 and the literature, most students reported enjoying 
the use of clickers in the classroom (Caldwell, 2007; Judson & Sawada, 2002), regardless of 
clicker or tenn test performance. 


General Discussion 

The results of the two studies confirm that clicker performance is highly correlated with 
course grades and that the relationship is maintained even after partialling out past academic 
performance, clicker participation, self-efficacy (Study 1 only), and sex. Establishing the 
clicker-course grades relationship is important, as it is apparent that by providing meaningful 
formative feedback to students, we may have a mechanism to improve student success if we can 
find an approach that will get them to seek help or alter other learning strategies when necessary. 

Although students reported making changes in study strategies based on their clicker 
performance, these reported changes did not generally relate to their actual clicker performance 
or their perfonnance on the term test. This is a significant finding, as one of the assumptions 
underlying clicker use in the classroom is that students will act on feedback received in class to 
make changes in their learning strategies which, in turn, will improve performance (Bruff, 2009). 
The formative feedback received both in class and via e-mail was not sufficient to cause most 
students to seek help or successfully alter their learning strategies. Even though the e-mail 
intervention did appear to influence some students who performed poorly in class to change their 
learning strategies, overall, no difference was found between those who changed their learning 
strategies and those who did not. It seems possible that e-mail may be effective in getting some 
students to pay attention to their perfonnance but is not sufficient to ensure a successful change 
in their approaches to learning. 

Svinicki (2004) has suggested that postsecondary students tend to utilize only one or two 
learning strategies, regardless of the task at hand. Furthermore, she states that their response to a 
new learning task is to try harder, rather than modifying their strategies. It may be that students’ 
reported increases in learning strategy use in Study 2 reflect their efforts to try harder, rather than 
using a new strategy or more effectively using an existing strategy. Future studies might wish to 
investigate more explicitly whether students take a surface (minimal efforts to meet course 
requirements) or deep (more achievement or strategic orientation) approach to learning, perhaps 
by utilizing the Lancaster Inventory of Approaches to Learning (Entwistle, Hanley, & Hounsell, 
1979). Biggs and Tang (2007) suggest that it is possible to shift students’ approach to learning, 
but that both teaching and assessment approaches may lead them to feel a surface approach is 
sufficient to meet their goals. Biggs and Tang (2007) suggest instructors may need to change 
their teaching methods to focus on activities that lead to deep learning, such as emphasizing 
depth rather than breadth in course coverage. Investigating how faculty encourage deep learning 
through their teaching practices is another avenue of research that could be investigated. 
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Although students reported changing their learning strategy use as a result of the clickers, 
it would be necessary to monitor their behaviour, or at least have students record in which 
specific activities they are engaged, to determine if they were truly changing their strategies 
based on the clicker feedback and if they were effective in using the changed strategies. One of 
the new social networking tools, such as Twitter, might be an easy way for students to record 
their own behaviour on a daily basis and for researchers to explore the clicker performance¬ 
learning strategy use relationship more thoroughly. 

In Study 2, participants’ self-regulation was related to their increases in self-reported 
learning strategy use for five of the eight strategies (i.e., the higher participants scored on self¬ 
regulation, the more likely they were to increase their use of learning strategies based on their 
clicker use), but, as outlined above, strategy use was not related to performance on the term test. 
Students are not necessarily skilled at metacognition and therefore may not be able to monitor 
their own comprehension effectively (Zimmerman & Paulsen, 1995). It has been suggested that 
when students have inaccurate perceptions of their self-efficacy or when they are too optimistic 
about their level of preparedness for tests, they may adopt a rather casual attitude to studying 
(Zimmerman & Paulsen, 1995). Given the high entrance average of many students in our studies 
(mean admission average was 88% for Study 1 and 89% for Study 2), it is perhaps not surprising 
that students may not have been as careful about their formal self-monitoring as they need to be. 
Research suggests that students can be taught to be self-regulated learners (e..g, Pintrich, 1995), 
and Coppola (1995) recommends numerous practices in this regard. They include using 
counterintuitive examples, so that students are forced to confront their own misconceptions, 
providing heuristics for learning and using think-aloud strategies or cognitive modelling. Also, 
Bruff (2009) has suggested that instructors give differential points based on how confident 
students are in their clicker responses. This method forces students to assess their understanding 
of their answers and may help develop their metacognive self-awareness. All of these practices 
are well suited to the clicker classroom and may help students to improve their performance. 

It was also somewhat surprising that students seldom reported asking their instructors or 
TAs for help, but more often relied on peers for learning support. Despite the fact that peers may 
be a good choice for solving some learning difficulties, it would also seem important for students 
to realize the critical role faculty and graduate teaching assistants may play in supporting their 
academic success (Pascarella & Terenzini, 2005). It appears that, at least in our first-year biology 
classes, students are unlikely to approach those who might most positively influence their 
learning. Even though students often believe that they will get the best help from the instructor, 
they still turn to peers; reasons given include not wanting to appear unintelligent in front of a 
teacher and feeling that friends have an obligation to help. Also, turning to peers or visiting the 
biology Web page (Study 2) may simply be more convenient (Ryan & Pintrich, 1998). 

The ability of clickers to impact on test perfonnance will depend upon students using 
their clicker perfonnance to change their study strategies successfully. However, this will only 
lead to an increase in test grades if there is a strong congruence between clicker questions asked 
in class and questions asked on exams (Carnaghan & Webb, 2005). It is interesting to note that 
students in Barnett’s (2006) study claimed to have a preference for being asked conceptual and 
diagnostic questions, rather than factual questions. Particularly if an instructor is interested in 
using active teaching strategies such as peer instruction with the clickers, it appears to be 
important that students are asked questions at a deeper level than simple factual questions. Only 
deep-level questions are likely to lead to good discussion in class (Biggs & Tang, 2007; Svinicki, 
2004). Caldwell (2007) provides suggestions about how to design good clicker questions. Of 
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particular interest to science instructors would be her recommendation to create questions that 
focus on misconceptions and answers that focus on common errors. Given students’ well- kn own 
tendency to hold on to misconceptions in science (Smith, diSessa, & Roschelle, 1993), even after 
class instruction, this may provide a technique for students to begin to address this problem in 
comprehension. 

Furthermore, the results of this research support the finding that students rate clickers 
positively (Caldwell, 2007; Judson & Sawada, 2002) and do so independently of their 
performance on clicker and test questions. Although this might seem a trivial finding, it is 
important, particularly for science students, as many report not liking science (Duncan, 2005). 
Therefore finding strategies, such as using clickers, to engage students in their learning seems 
paramount in encouraging science students to go forward in the discipline (Kaleta & Joosten, 
2007). 

One possible limitation to these studies is that students in both studies scored 
approximately 4% to 5% higher on the tests than the classes as a whole. It may be that a third 
variable, such as motivation, predicts higher engagement, which leads to both higher grades and 
increased likelihood of participation in research. Even though the difference is not dramatic, the 
relatively higher academic performance of the participants may be a factor in generalizing these 
results. 

In summary, our two studies add substantially to the literature on the use of clickers in 
the classroom by demonstrating that clicker performance does predict students’ test and course 
performance. Unfortunately, students do not generally alter their help seeking or other learning 
strategies based on their actual clicker performance (even though they report making changes 
based on the clickers). Further, when participants report making changes to their learning 
strategy use, the changes were not generally related to their performance in the course. It may be 
that students need to be taught more explicitly about metacognitive self-regulation and effective 
learning strategy use if they are to be successful in improving their academic performance. As 
Pintrich and Zusho (2007) have suggested, first-year students may not be aware of the different 
learning strategies that they may use. They may also need to adapt their learning strategies from 
the high school environment to the more complex university milieu. This lack of knowledge of 
effective learning strategies may be one reason they do not seek help from learning services, as 
they may be unaware of the impact of different learning strategies on their achievement. Clickers 
play an important role in informing students about their own learning in the classroom but, to be 
truly effective, instructors may need to teach students how and why self-regulation and effective 
use of learning strategies are essential for academic success at university. 
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